# Visual Reasoning

QVQ-Max is a visual reasoning model launched by the Qwen team, capable of understanding and analyzing image and video content to provide solutions. It is not limited to text input but can also handle complex visual information. Suitable for users who need multi-modal information processing, such as in education, work, and life scenarios. This product is developed based on deep learning and computer vision technology and is suitable for students, professionals, and creative individuals. This is the initial release, and subsequent optimizations will be continuous.

Aya Vision 32B is an advanced vision-language model developed by Cohere For AI, boasting 32 billion parameters and supporting 23 languages, including English, Chinese, and Arabic. This model combines the latest multilingual language model Aya Expanse 32B and the SigLIP2 vision encoder, achieving visual and language understanding integration through a multimodal adapter. It excels in the vision-language field, capable of handling complex image and text tasks such as OCR, image captioning, and visual reasoning. The release of this model aims to promote the popularization of multimodal research, providing a powerful tool for global researchers with its open-source weights. The model is licensed under CC-BY-NC and is subject to Cohere For AI's fair use policy.

AlphaMaze is a project focused on enhancing the visual reasoning abilities of Large Language Models (LLMs). It trains models through maze tasks described in text format, enabling them to understand and plan in spatial structures. This method avoids complex image processing and directly assesses the model's spatial understanding through text descriptions. Its main advantage is the ability to reveal how the model thinks about spatial problems, rather than simply whether it can solve them. The model is based on open-source frameworks and aims to promote research and development of language models in the field of visual reasoning.

AlphaMaze is a decoder language model designed specifically for solving visual reasoning tasks. It demonstrates the potential of language models in visual reasoning through training on maze-solving tasks. The model is built upon the 1.5 billion parameter Qwen model and is trained with Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Its main advantage lies in its ability to transform visual tasks into text format for reasoning, thereby compensating for the lack of spatial understanding in traditional language models. The development background of this model is to improve AI performance in visual tasks, especially in scenarios requiring step-by-step reasoning. Currently, AlphaMaze is a research project, and its commercial pricing and market positioning have not yet been clearly defined.

QVQ-72B-Preview is an experimental research model developed by the Qwen team, focusing on enhancing visual reasoning capabilities. The model demonstrates strong abilities in multidisciplinary understanding and reasoning, achieving significant advances especially in mathematical reasoning tasks. Although advancements have been made in visual reasoning, it does not completely replace the capabilities of Qwen2-VL-72B, and may gradually lose focus on image content in multi-step visual reasoning, leading to hallucinations. Furthermore, QVQ does not show significantly better performance in basic recognition tasks compared to Qwen2-VL-72B.

Claude 3.5 Sonnet, developed by Anthropic, strikes a remarkable balance between intelligence, speed, and cost. This model sets new industry benchmarks in graduate-level reasoning, undergraduate-level knowledge, and programming proficiency. It excels at understanding nuances, humor, and complex instructions, and can generate high-quality content in a natural and friendly tone. Additionally, it demonstrates strong capabilities in visual reasoning, chart interpretation, and image-to-text transcription, making it an ideal choice for industries like retail, logistics, and financial services.

Visual Sketchpad is a framework that provides a visual sketchpad and drawing tools for multimodal large language models (LLMs). It allows models to operate on visually created elements while planning and reasoning, unlike previous methods that relied solely on text for reasoning steps. Visual Sketchpad enables models to draw using lines, boxes, annotations, and other more human-like drawing elements, thereby facilitating better reasoning. Additionally, it can incorporate expert vision models, such as object detection models for drawing bounding boxes or segmentation models for drawing masks, to further enhance visual perception and reasoning capabilities.

Cantor is a multimodal chain-of-thought (CoT) framework that leverages a perception-decision architecture to combine visual context acquisition with logical reasoning, effectively solving complex visual reasoning tasks. Acting as a decision generator, Cantor integrates visual input to analyze images and questions, ensuring tighter alignment with real-world scenarios. Furthermore, Cantor utilizes the advanced cognitive capabilities of large language models (LLMs) as multi-faceted experts to deduce higher-level information, enriching the CoT generation process. Extensive experiments on two challenging visual reasoning datasets demonstrate the effectiveness of the proposed framework. Notably, Cantor achieves significant improvements in multimodal CoT performance without requiring fine-tuning or real-world reasoning, surpassing existing baselines."

Cola is a method that uses a language model (LM) to aggregate the outputs of 2 or more vision-language models (VLMs). Our model assembly method is called Cola (COordinative LAnguage model or visual reasoning). Cola performs best when the LM is fine-tuned (called Cola-FT). Cola is also effective in zero-shot or few-shot context learning (called Cola-Zero). In addition to performance improvements, Cola is also more robust to VLM errors. We demonstrate that Cola can be applied to various VLMs (including large multimodal models like InstructBLIP) and 7 datasets (VQA v2, OK-VQA, A-OKVQA, e-SNLI-VE, VSR, CLEVR, GQA), and it consistently improves performance.